I do want to slim this down further, but not change too much to make it too much harder to use.

The rough plan was to install docker tooling inside this container and sharing the docker.sock allowing for us to have desktop shortcuts for running docker exec into other containers, and I propose we should configure that with docker labels.

@marc-hanheide do we want to rebase this onto the main branch and merge this after #1?

I do want to slim this down further, but not change too much to make it too much harder to use.

The rough plan was to install docker tooling inside this container and sharing the docker.sock allowing for us to have desktop shortcuts for running docker exec into other containers, and I propose we should configure that with docker labels.

I'm not convinced by making an assumption of access to docker socket. That's a deployment question that shouldn't have an impact on the container.

we can use devcontainer labels to facilitate attaching to it

@marc-hanheide do we want to rebase this onto the main branch and merge this after #1?

yes, feel free to merge #1 if happy with it and bring it all together

then my cuda_desktop should be depend on your ros_vnc flavours, to offer a full containeriesed ROS2 desktop experience, with rviz2 ready to go. So, I suggest the following pipelines:

ros (or ros_cuda) -> ros_vnc (or ros_cuda_vnc) -. ros_desktop (or ros_cuda_desktop)

with the *_cuda_* flavours being much bigger, but providing EGL/VGL acceleration utilising the host’s GPU.

I understand that exposing the Docker socket is something we cannot rely on to access other containers terminals. But this was an idea on how to implement the user having access to spawn items from other containers inside the web view.

I am now considering the purpose of accessing anything other than running a display inside this container, or to put it simply, why/when does a user/developer need to launch any apps inside the container?

I am a little concerned about implementing the vnc container on top of a ROS enabled base with a minimal ROS environment. As if the display container is only being used as a x11 destination, this will then make the container larger than it needs to be (and have more deps).

Unless what you're considering is to have this as an alternative base for using with development allows for a similar experience to what is inside LCAS/docker_cuda_desktop.

From my perspective that is adding a lot more complexity when it comes to real world deployment as you're using a completely different environment from deployment. Should the developer be running the vnc container as a part of a compose stack inside VS Code?

I think it would be best to work through to an MVP, so what I am thinking and my concept can be fully demoed. Then we can go from there and decide which is the best way forward.

I understand that exposing the Docker socket is something we cannot rely on to access other containers terminals. But this was an idea on how to implement the user having access to spawn items from other containers inside the web view.

I am now considering the purpose of accessing anything other than running a display inside this container, or to put it simply, why/when does a user/developer need to launch any apps inside the container?

I am a little concerned about implementing the vnc container on top of a ROS enabled base with a minimal ROS environment. As if the display container is only being used as a x11 destination, this will then make the container larger than it needs to be (and have more deps).

Unless what you're considering is to have this as an alternative base for using with development allows for a similar experience to what is inside LCAS/docker_cuda_desktop.

From my perspective that is adding a lot more complexity when it comes to real world deployment as you're using a completely different environment from deployment. Should the developer be running the vnc container as a part of a compose stack inside VS Code?

I think it would be best to work through to an MVP, so what I am thinking and my concept can be fully demoed. Then we can go from there and decide which is the best way forward.

OK, I think I understand (maybe). You want “just” an X Server that other containers can connect to (via normal X protocol?), So, they may set export DISPLAY=vnc:0? Is that the idea? In that case all the other containers still need to have all the things for acting as an X client installed. So they all would contain a lot of X-related libs, wouldn’t they?

That is very different to what I had in mind, indeed then. To clarify what I had in mind:

• A base line ROS container (and one with CUDA)
• A lean add-on on top of that that is a GUI-enabled ROS container image. Basically, just adds VGL, turbovnc and novnc. (will not add more than a few Mb), the CUDA flavour is OpenGL accelerated through VGL (as is currently the case)

Then whenever we build bespoke images they are built on top of either these flavours.

E.g., I would suggest a dedicated “rviz”/rqt image, build on top of CUDA-enabled VNC. This can then be used in many instances to introspect another container

But, by all means, complete a MVP as I’m still not sure I fully get it.

Any update @cooperj ?

Hey @marc-hanheide this is ready for some more oversight, I have working demo for you to look at:

If you want to replicate this, this is the following structure you need:

├── aoc_container_base
│   ├── base.dockerfile
│   ├── cuda_desktop.dockerfile
│   ├── cuda.dockerfile
│   ├── docker
│   ├── LICENSE
│   ├── README.md
│   └── vnc.dockerfile
├── aoc_hri
│   ├── build
│   ├── db
│   ├── docs
│   ├── install
│   ├── log
│   ├── README.md
│   └── src
├── aoc_robot_simulator
│   ├── contrib
│   ├── README.md
│   ├── robots
│   └── src
└──── compose.yaml

services:
  vnc:
    build: 
      context: ./aoc_container_base
      dockerfile: vnc.dockerfile
    user: "lcas" 
    ports:
      - "5801:5801"
    volumes:
      - x11:/tmp/.X11-unix
      - /var/run/docker.sock:/var/run/docker.sock
    devices:
      - /dev/dri:/dev/dri  # GPU access for VirtualGL
    shm_size: '2gb'
    stdin_open: true
    tty: true
    networks:
      - aoc_ros
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu, compute, utility, graphics]
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=all

  simulator:
    build: 
      context: ./aoc_robot_simulator
      dockerfile: .devcontainer/Dockerfile
    user: "ros"
    runtime: nvidia
    volumes:
      - x11:/tmp/.X11-unix
    depends_on:
      - vnc
    stdin_open: true
    tty: true
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu, compute, utility, graphics]
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=all
      - DISPLAY=:1.0
      - ROS_DOMAIN_ID=1
      - ROBOT_MODEL=hunter
      - ROBOT_NAME=hunter_001
      - USE_SIM=true
      - WORLD=person_walking
    networks:
      - aoc_ros
      
  hri:
    build: 
      context: ./aoc_hri
      dockerfile: .devcontainer/Dockerfile
    user: "ros" 
    runtime: nvidia
    stdin_open: true
    tty: true
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu, compute, utility, graphics]
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=all
      - DISPLAY=:1.0
      - USE_SIM=true
    volumes:
      - x11:/tmp/.X11-unix
    depends_on:
      - vnc
      - simulator
    networks:
        - aoc_ros

volumes:
  x11:

networks:
  aoc_ros:

Questions to answer:

• Do we want to have docker or a terminal available in the noVNC session?
• • Is this a value add?
• How do we want to work out the GH workflows, should they be adapted to support the VNC container or should that container be built seperately?
• • How about the name lcas.lincoln.ac.uk/vnc:1 (where 1 is tagged release number)?

I propose all ROS work should happen in a separate container, and if required all debugging tasks should be ran in a separate container which is connected to both the network (so can get ros topics and data) and the x11 socket.

Would this debugging container take the role of being the terminals? Do we provide that with code server?

@cooperj I've opened a new pull request, #7, to work on those changes. Once the pull request is ready, I'll request review from you.

Regarding authentication and listen addresses, I did spend some time trying to play with getting this working previously.

The way I see copilot suggesting is letting the dev have an option of setting a password on the noVNC interface which is disabled by default.

What I think we should do is the following options:

1. If we don't need to auth (i.e. student facing robot) - we don't need it!
2. If we are deploying with VPN access into the robot, have in the compose file the port binding tagged to the VPN interface (i.e. 10.8.0.123:5801:5801 where 10.8.0.123 is the IP on VPN)
3. Sit the robot interface(s) behind a reverse proxy and put auth on that, this can be anything we like from nice OIDC to basic auth.

I feel like we should be going in the order of preference to 2->1->3, with options 1 and 2 being the easiest to implement.